See copyright notice at the bottom of this page.
List of All Posters
How are Runs Really Created
August 14, 2002 - Arvid Engen
Okay, I'm confused.
Tango, if the goal of the work you describe here is in fact to derive optimal situation-neutral values for each batting event, then what really distingishies it from Extrapolated Runs, or another linear-weights based method?
It seems that you're going to wind up with coefficients that, while perhaps more defensible aesthetically, are less valid empirically than those derived via linear regression. You can't beat linear regression, baby.
How are Runs Really Created - Third Installment
September 16, 2002 - Arvid Engen
*** First of all, this is not true. Generally speaking, those events ARE independent of the HR. I ran further studies that controlled for those events (for example, looked only at games with 2 to 4 walks, 6 to 8 singles, etc, and separated them by the HR class) so that there was virtually no difference in those events. The results were the same.
I'm not convinced. Do you mean to suggest that there's no meaningful correlation between (say) the number of doubles hit in a game and the number of home runs? That's a highly counterintuitive conclusion.
I'm also not certain that the experiment you've described would do an adequate job of controlling for those effects. "2 to 4 walks" is a fairly wide range, the difference between Roy Oswalt on the one hand, and Matt Clement on the other. Moreover, you're not controlling for the interactions between these run elements. One more double or two more singles or two more walks might not make that much difference, but one more double and two more singles and two more walks certainly would.
*** In extreme examples, you can't hide the shortcomings of your models or estimates. They stick out like a sore thumb. And note, in my extreme examples, the dataset went only so far as Barry Bonds' 01/02. So, it was not "unrealistic" extreme, but realistic extremes.
....
*** As I said, almost all run evaluators are similar at the .300 to .400 OBA range. This will help you determine the true value of those extreme players that GMs are trying to figure if they are overpaying them.
The competing models aren't trying to deal with extreme cases. To vigorously pat yourself on the back because BaseRuns specifically does better what it's specifically designed to do better is a bit facetious. If BaseRuns is 50% more accurate at dealing with extreme cases, and 1% less accurate at dealing with realistic cases (I don't know that it is), that seems to me like one step forward and two back.
Yes. This series of articles explains a team of Barry Bonds, a team of Pedros (i.e., Pedro himself), and virtually any run environment, regardless of whether that run environment is due to the hitter, the pitcher, the fielders, or the park.
But how about one Barry Bonds and eight mortals? What I'd like to see is a comparison of the systems within an actual major league context, not a simulation that you've designed to produce an outcome that is preordained to be favorable to your cause.
What is the question you're trying to answer? What are the implications of your research? I apologize in advance for my confrontational tenor, but your advocacy of BaseRuns comes across as almost cultish, based on a series of assumptions it conflates with Truth, without regard for the world around it. It is like a sabermetric version of Ayn Rand.
How are Runs Really Created - Third Installment
September 17, 2002 - Arvid Engen
1. I will try and tone things down a bit, but think of this as the sort of tough love you'd encounter in defending a dissertation. It is clear that all of this makes sense to you, but it is not so transparent to a well-informed audience.
I also think that you're inviting somewhat more ... aggressive feedback when you say things like "Runs Created is dead, BaseRuns is the now", or invoke (incorrectly) something like the Heisenberg Uncertainty Principle. I mean, you're talking the talk.
2. Thank you for presenting the table of correlations.
3. The fundamental point is that there's no "Holy Grail" here. Runs are created by the particular combination of batting and baserunning events in a particular inning of a particular game. Any attempt to generalize these unique sequences of events into something universally applicable has got to make approximations and assumptions.
Linear weights cuts a different corner than BaseRuns does, by focusing on data at the season level rather than at the game level. You assert that focusing on data at the game level is True, without presenting evidence either as to the utility of this approach (what would Billy Beane do with it?), or to its aesthetic purity. Why focus on the game level, rather than at the inning level? Why try and take all of the context out of run creation at all?
4. My point in criticising your experimental design is that the coefficients you use in BaseRuns were derived based on data gathered at the game level, and that you then use game-level data in order to test its superiority. If you tested the various systems based on season-level data, linear weights would triumph, because that's how it is derived.
5. In the OBA chart you present above, BaseRuns is considerably less accurate over the entire normal range of OBA's. Missing by an additional .25 runs per game would amount to about 40 runs or 4-5 games over the course of a season. That is not "very very similar"; you have made a substantial trade-off here!
6. I was disappointed that you did not reply to the Ayn Rand ad hominem! I suspect that you have a tattered and dog-eared copy of The Fountainhead sitting on your bedside table.
How are Runs Really Created - Third Installment
September 17, 2002 - Arvid Engen
Tango,
Thank you for your follow-up comments; I don't mean for it to come across as though I think what you've done is without merit. However a couple of further questions/comments:
1. It seems almost oxymoronic that BaseRuns doesn't do particuarly well relative to different levels of OBP, but does do very well relative to different levels of OPS. This suggests to me that there is some sort of interaction between the "getting on base" element and the "moving runners along" element that has been lost in the attempt to segregate those two things from one another. For one thing, the probabilities of particular batting outcomes aren't independent of the bases occupied during a given plate appearance.
2. I suppose I'm still somewhat put off by the implications that BaseRuns is a "true" or "real" or otherwise aesthetically pure measure of run creation. Even if you look at data on an inning-by-inning basis, it is still an approximation:
BB-1B-1B-HR-K-K-K produces 4 runs, whereas HR-K-K-1B-1B-BB-K normally produces 1.
That, in aggregating data to the season level, unusual and random sequences tend to get lost in the noise, is as much an advantage as a disadvantage.
How are Runs Really Created - Third Installment
September 17, 2002 - Arvid Engen
Dear Tony Eason,
Excuse me for wearing white after labor day. I spend too much time worrying about the nuances of baseball statistics as it is, and I choose to focus on this forum for a variety of reasons. BaseRuns certainly isn't widely accepted to the point where its assertions can be taken at face value without further validation; if it were, Tango would not have had to "spread the gospel" over at this site.
The peer review process, if you will, is still ongoing. I don't doubt that you and Tango have worked with the data extensively, have invested significant time in improving the system, but I am not convinced that the philosophical underpinnings of the system have been well articulated, here or anywhere else.
runs=baserunners*%whoscore+HR isn't anything more than an identity. For it to be of any use, you need to solve for its parameters based on the same dirty, polluted dataset that all the other methods have used. It is not clear to me that this particular formulation represents anything other than an algebraic rearrangement of a linear weights formula. You arrive at different coefficients because you use game-based data, instead of season-based data, but there is nothing in the identity itself that is particularly amenable to that technique. You could just as easily generate linear weights or extrapolated runs or whatever else using game-based data.
There is also no intrinsic relationship between the BaseRuns formula and the choice of static versus dynamic LW coefficients. If we wanted LW coefficients that correspond to a high-offense environment, we could get plenty good results by limiting our regression to high-offense seasons. It is not clear to me that BaseRuns has reached some sort of equilibrium between the utility of a formula that can facilitate comparison of offensive performance across different contexts, and the reality that offensive output is inherently context-bound.
BaseRuns is simply a refinement of linear weights. A tremendous amount of this discussion consists of age old questions couched in a new vocabulary. That does mean that these questions are resolved to everyone's satisfaction, or that the research this project has contributed isn't worthwhile, and I apologize if I have implied otherwise. It does mean that you guys ought to be careful not to mistake the novelty of your jargon with the novelty of your ideas. Certainly, some of the language you and Tango have invoked in defending the system suggests that this may be a strong possibility.
For Tango's next article, I would suggest that he discuss the offensive performance of the 2001 San Francisco Giants in terms of BaseRuns.
How are Runs Really Created - Third Installment
September 18, 2002 - Arvid Engen
Finally, in the calculation of this ratio as presented in the article ... the calculation of B is no more complicated than that used by other run estimators which apply coefficients to each of these events. Because these coefficients are determined in a somewhat similar manner to other estimators, from real world data, they are also estimates and might be likely to introduce some degree of error.
Is there any reason to believe that such estimates are subject to any less degree of error than are the other run estimators?
That's why I say that this is an algebraic rearrangement of a traditional run estimation formula; you're calculating the same damned coefficients, and adding some window dressing. Well, not quite; you're also introducing a constraint or two into the calculation, the effect of which seems to be:
1. More robustness across different run-scoring environments. 2. Somewhat less accuracy in normal run-scoring environments.
But unlike some other estimators, these coefficients do not represent the VALUE of these events. Instead, they represent only the impact on the calculation of the overall rate of succesfully scoring baserunners.
That sounds neat, but that's little more than a semantic distinction. The "value" of the coefficients in a LW formula is the estimated impact of a particular event on scoring runs. The "value" of the coefficients in the BaseRuns formula is the estimated impact of a particular event on scoring baserunners. I don't see how the latter is superior to the former in any meaningful way.
Copyright notice
Comments on this page were made by person(s) with the same handle, in various comments areas, following Tangotiger © material, on Baseball Primer. All content on this page remain the sole copyright of the author of those comments.
If you are the author, and you wish to have these comments removed from this site, please send me an email (tangotiger@yahoo.com), along with (1) the URL of this page, and (2) a statement that you are in fact the author of all comments on this page, and I will promptly remove them.